<!DOCTYPE HTML>
<html>
  <head>
    <title>MediaSource API</title>
    <link rel="stylesheet" href="video-working-draft.css">
    <link rel="stylesheet" href="main.css">
  </head>
  <body>
    <div class="head">
      <h1>MediaSource API</h1>
      <h2>Draft Proposal</h2>
      <dl><dt>Editor:</dt><dd>Aaron Colwell, Google Inc.&lt;acolwell@google.com&gt;</dd></dl>
    </div>
    <h2>Status of this Document</h2>
    <p>This document is a draft specification proposal with no official status. Send comments to  <a href="mailto:acolwell@google.com">Aaron Colwell</a>. It is inappropriate to cite this document except as a work in progress.</p>

    <h2>Abstract</h2>
    <p>
      This proposal introduces a MediaSource object which allows 
      JavaScript to generate media streams for playback in HTMLMediaElements. 
      Allowing JavaScript to generate streams facilitates a variety of use 
      cases like adaptive streaming and time shifting live streams.
    </p>

    <h2>Table of Contents</h2>

    <ol id="toc">
      <li><a href="#introduction">1. Introduction</a></li>
      <li><a href="#mediasource">2. MediaSource Interface</a>
      <li><a href="#event-summary">3. Event Summary</a></li>
      <li><a href="#algorithms">4. Algorithms</a></li>
        <ol>
          <li><a href="#algorithms-attach">4.1 Attaching to a media element</a></li>
          <li><a href="#algorithms-detach">4.2 Detaching from a media element</a></li>
          <li><a href="#waiting-timeout">4.3 Waiting Timeout</a></li>
          <li><a href="#seeking">4.4 Seeking</a></li>
        </ol>
      </li>
      <li><a href="#examples">5. Examples</a></li>
    </ol>

    <h2 id="introduction">1. Introduction</h2>
    <p>Historically &lt;audio&gt; and &lt;video&gt; have only allowed media playback from container file formats. Support for use cases like live have required
    hacks to the container format, like 0 size segments in WebM, or required the browser to support manifest formats like <a href="http://tools.ietf.org/html/draft-pantos-http-live-streaming-06">HLS</a>. Supporting new adaptive streaming proposals like <a href="http://www.3gpp.org/ftp/Specs/html-info/26247.htm">3GP-DASH</a> also require significant browser changes and the adaptation algorithms cannot be changed once the browser is released. The file container model also puts constraints on on-demand playback. Efficient seek support requires an index to be transfered to the browser which results in either delayed startup, or delays during the first seek. All of these issues stem from the fact that the browser is expected to manage all media fetching and do so based on the information contained in a single URL.</p>
    <p>This proposal introduces a new object called MediaSource that allows JavaScript to control how media data is fetched. This will enable significant flexibility for media fetching without requiring browser changes. Manifest formats like HLS & 3GP-DASH could be handled in JavaScript. Indexes for on-demand playback would not be needed because JavaScript could make an AJAX call to a web service to find an appropriate seek point. Adaptive streaming algorithms could be implemented and evolved over time without needing to upgrade the browser. Fetching media from multiple CDNs or splicing in media from the local file store could also be implemented without browser changes. Allowing JavaScript to control media stream construction opens may avenues for experimentation and innovation.</p>

    <h2 id="mediasource">2. MediaSource Interface</h2>
    <p>This interface provides a way to stream data to a media element.
    </p>
    <pre class="idl">
[<a href="#dom-mediasource" title="dom-mediasource" id="dfnReturnLink-0">Constructor</a>(in DOMString type)]
interface MediaSource {
  readonly attribute DOMString <a href="#dom-mediasource-type" title="dom-MediaSource-type">type</a>;

  // states
  const unsigned short <a href="#dom-mediasource-closed" title="dom-MediaSource-CLOSED">CLOSED</a> = 0;
  const unsigned short <a href="#dom-mediasource-open" title="dom-MediaSource-OPEN">OPEN</a> = 1;
  const unsigned short <a href="#dom-mediasource-ended" title="dom-MediaSource-ENDED">ENDED</a> = 2;

  readonly attribute unsigned short <a href="#dom-mediasource-readystate" title="dom-MediaSource-readyState">readyState</a>;
  
  readonly attribute unsigned long <a href="#dom-mediasource-bytesavailable" title="dom-MediaSource-bytesAvailable">bytesAvailable</a>;

  bool <a href="#dom-mediasource-append" title="dom-MediaSource-append">append</a>(in ArrayBuffer data);
  void <a href="#dom-mediasource-endofstream" title="dom-MediaSource-endOfStream">endOfStream</a>();

  // event handler attributes
  attribute Function <a href="#dom-mediasource-onopen" title="dom-MediaSource-onopen">onopen</a>;
  attribute Function <a href="#dom-mediasource-onwaiting" title="dom-MediaSource-onwaiting">onwaiting</a>;
  attribute Function <a href="#dom-mediasource-onflush" title="dom-MediaSource-onflush">onflush</a>;
  attribute Function <a href="#dom-mediasource-onprogress" title="dom-MediaSource-onprogress">onprogress</a>;
  attribute Function <a href="#dom-mediasource-onended" title="dom-MediaSource-onended">onended</a>;
  attribute Function <a href="#dom-mediasource-onclose" title="dom-MediaSource-onclose">onclose</a>;
};
MediaSource implements EventTarget;

    </pre>

    <p>The <dfn id="dom-mediasource" title="dom-mediasource"><code>MediaSource(<var title>type</var>)</code></dfn> constructor takes a single argument. This argument, <var title>type</var>, specifies the mimetype for data that will be streamed to the media element. This is similar to the type attribute on a &lt;source&gt; element. If <var title>type</var> is a null or empty string a <code>SYNTAX_ERR</code> exception will be thrown. 
    </p>

    <p>The <dfn id="dom-mediasource-type" title="dom-mediasource-type"><code>type</code></dfn> attribute returns the type value passed into the constructor.
    </p>

    <p>The <dfn id="dom-mediasource-readystate" title="dom-mediasource-readyState"><code>readyState</code></dfn> attribute represents the state of the source. It can have the following values:
    </p>
    <dl>
      <dt><dfn id="dom-mediasource-closed" title="dom-mediasource-CLOSED"><code>CLOSED</code></dfn> (numeric value 0)</dt>
      <dd>Indicates the source is not currently attached to a media element</dd>

      <dt><dfn id="dom-mediasource-open" title="dom-mediasource-OPEN"><code>OPEN</code></dfn> (numeric value 1)</dt>
      <dd>The source has been opened by a media element.</dd>

      <dt><dfn id="dom-mediasource-ended" title="dom-mediasource-ENDED"><code>ENDED</code></dfn> (numeric value 2)</dt>
      <dd>endOfStream() has been called on this source. No further append() calls are allowed in this state.</dd>
    </dl>
    <p>When the object is created <code><a href="#dom-mediasource-readystate" title="dom-MediaSource-readyState">readyState</a></code> must be set to <code><a href="#dom-mediasource-closed" title="dom-MediaSource-CLOSED">CLOSED</a></code> (0).
    </p>

    <p>The <dfn id="dom-mediasource-bytesavailable" title="dom-mediasource-bytesAvailable"><code>bytesAvailable</code></dfn> attribute indicates the current number of bytes available in the source buffer. append() will fail if called with an ArrayBuffer larger than this value.
    </p>

    <p>The <dfn id="dom-mediasource-append" title="dom-mediasource-append"><code>append(<var title>data</var>)</code></dfn> method must run the following steps:</p>
    <ol>
      <li>If the first argument is null or <var title>data</var>.byteLength is 0 then throw an <code>INVALID_ACCESS_ERR</code> exception and abort these steps.</li>
      <li>If the <code><a href="#dom-mediasource-readystate" title="dom-MediaSource-readyState">readyState</a></code> attribute is not in the <code><a href="#dom-mediasource-open" title="dom-MediaSource-OPEN">OPEN</a></code> state then throw an <code>INVALID_STATE_ERR</code> exception and abort these steps.</li>
      <li>
        <p>Run the first matching steps from the following list:</p>
        <dl class="switch">
          <dt>If <var title>data</var>.byteLength is greater than the <code><a href="#dom-mediasource-bytesavailable" title="dom-MediaSource-bytesAvailable">bytesAvailable</a></code> attribute</dt>
          <dd><p>Return <code>false</code>.</p></dd>
          <dt>Otherwise</dt>
          <dd>
            <p>Copy contents of <var title>data</var> into the media element's decode buffer.</p>
            <p>Decrement <code><a href="#dom-mediasource-bytesavailable" title="dom-MediaSource-bytesAvailable">bytesAvailable</a></code> attribute by <var title>data</var>.byteLength.</p>
            <p>Return <code>true</code>.</p>
          </dd>
        </dl>
      </li>
    </ol>

    <p>The <dfn id="dom-mediasource-endofstream" title="dom-mediasource-endOfStream"><code>endOfStream()</code></dfn> method must run the following steps:</p>
    <ol>
      <li>If the <code><a href="#dom-mediasource-readystate" title="dom-MediaSource-readyState">readyState</a></code> attribute is not in the <code><a href="#dom-mediasource-open" title="dom-MediaSource-OPEN">OPEN</a></code> state then throw an <code>INVALID_STATE_ERR</code> exception and abort these steps.</li>
      <li>Change the <code><a href="#dom-mediasource-readystate" title="dom-MediaSource-readyState">readyState</a></code> attribute's value to <code><a href="#dom-mediasource-ended" title="dom-MediaSource-ENDED">ENDED</a></code>.</li>
      <li>Fire a simple event named <code><a href="#dom-mediasource-endedevent" title="dom-MediaSource-endedEvent">ended</a></code> at the source.
      <li>Notify the media element that the end of stream has been reached.</li>
    </ol>

    <table>
      <thead>
        <tr>
          <th>Event handler</th>
          <th>Event handler event type</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <td><dfn id="dom-mediasource-onopen" title="dom-mediasource-onopen"><code>onopen</code></dfn></td>
          <td><code><a href="#dom-mediasource-openevent" title="dom-MediaSource-openEvent">open</a></code></td>
        </tr>
        <tr>
          <td><dfn id="dom-mediasource-onwaiting" title="dom-mediasource-onwaiting"><code>onwaiting</code></dfn></td>
          <td><code><a href="#dom-mediasource-waitingevent" title="dom-MediaSource-waitingEvent">waiting</a></code></td>
        </tr>
        <tr>
          <td><dfn id="dom-mediasource-onflush" title="dom-mediasource-onflush"><code>onflush</code></dfn></td>
          <td><code><a href="#dom-mediasource-flushevent" title="dom-MediaSource-flushEvent">flush</a></code></td>
        </tr>
        <tr>
          <td><dfn id="dom-mediasource-onprogress" title="dom-mediasource-onprogress"><code>onprogress</code></dfn></td>
          <td><code><a href="#dom-mediasource-progressevent" title="dom-MediaSource-progressEvent">progress</a></code></td>
        </tr>
        <tr>
          <td><dfn id="dom-mediasource-onended" title="dom-mediasource-onended"><code>onended</code></dfn></td>
          <td><code><a href="#dom-mediasource-endedevent" title="dom-MediaSource-endedEvent">ended</a></code></td>
        </tr>
        <tr>
          <td><dfn id="dom-mediasource-onclose" title="dom-mediasource-onclose"><code>onclose</code></dfn></td>
          <td><code><a href="#dom-mediasource-closeevent" title="dom-MediaSource-closeEvent">close</a></code></td>
        </tr>
      </tbody>
    </table>

    <h2 id="event-summary">3. Event Summary</h2>
    <table>
      <thead>
        <tr>
          <th>Event name</th>
          <th>Interface</th>
          <th>Dispatched when...</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <td><dfn id="dom-mediasource-openevent" title="dom-mediasource-openEvent"><code>open</code></dfn></td>
          <td><code>Event</code></td>
          <td>When the source transitions from <code><a href="#dom-mediasource-closed" title="dom-MediaSource-CLOSED">CLOSED</a></code> to <code><a href="#dom-mediasource-OPEN" title="dom-MediaSource-OPEN">OPEN</a></code></td>
        </tr>
        <tr>
          <td><dfn id="dom-mediasource-waitingevent" title="dom-mediasource-waitingEvent"><code>waiting</code></dfn></td>
          <td><code>Event</code></td>
          <td>When the media element has played all the data passed to <code><a href="#dom-mediasource-append" title="dom-MediaSource-append">append()</a></code> and needs more data to prevent playback from stalling.</td>
        </tr>
        <tr>
          <td><dfn id="dom-mediasource-flushevent" title="dom-mediasource-flushEvent"><code>flush</code></dfn></td>
          <td><code>Event</code></td>
          <td>When the media element initiates a seek.</td>
        </tr>
        <tr>
          <td><dfn id="dom-mediasource-progressevent" title="dom-mediasource-progressEvent"><code>progress</code></dfn></td>
          <td><code>Event</code></td>
          <td>Every 350ms(+/-200ms) or every time <code><a href="#dom-mediasource-bytesavailable" title="dom-MediaSource-bytesAvailable">bytesAvailable</a></code> increases, whichever is <em>least</em> frequent.</td>
        </tr>
        <tr>
          <td><dfn id="dom-mediasource-endedevent" title="dom-mediasource-endedEvent"><code>ended</code></dfn></td>
          <td><code>Event</code></td>
          <td>When the source transitions from <code><a href="#dom-mediasource-open" title="dom-MediaSource-OPEN">OPEN</a></code> to <code><a href="#dom-mediasource-ended" title="dom-MediaSource-ENDED">ENDED</a></code>.</td>
        </tr>
        <tr>
          <td><dfn id="dom-mediasource-closeevent" title="dom-mediasource-closeEvent"><code>close</code></dfn></td>
          <td><code>Event</code></td>
          <td>When the source transitions to <code><a href="#dom-mediasource-closed" title="dom-MediaSource-CLOSED">CLOSED</a></code></td>
        </tr>
      </tbody>
    </table>

    <h2 id="algorithms">4. Algorithms</h2>
    <h3 id="algorithms-attach">4.1 Attaching to a media element</h3>
    <ol>
      <li>Create a new <code><a href="#dom-mediasource" title="dom-MediaSource">MediaSource</a></code> object with <code>new</code></li>
      <li>Create a URL for the source using <code>URI.createObjectURL()</code></li>
      <li>Set the <code>src</code> attribute on a media element or the <code>src</code> attribute on a &lt;source&gt; element associated with a media element.</li>
      <li><p>When the media element attempts the <a href="http://www.w3.org/TR/html5/video.html#concept-media-load-resource">resource fetch algorithm</a> with the URL for the source object it will take one of the following actions:</p>
        <dl class="switch">
          <dt>If the source's <code><a href="#dom-mediasource-readystate" title="dom-MediaSource-readyState">readyState</a></code> is NOT set to <code><a href="#dom-mediasource-closed" title="dom-MediaSource-CLOSED">CLOSED</a></code></dt>
          <dd>Abort media element's <a href="http://www.w3.org/TR/html5/video.html#concept-media-load-resource">resource fetch algorithm</a> and run the steps to report a <code>MEDIA_ERR_SRC_NOT_SUPPORTED</code> error.</dd>
          <dt>If the source's <code><a href="#dom-mediasource-type" title="dom-MediaSource-type">type</a></code> attribute contains a mimetype that is not supported by the media element.</dt>
          <dd>Abort media element's <a href="http://www.w3.org/TR/html5/video.html#concept-media-load-resource">resource fetch algorithm</a> with an error.</dd>
          <dt>Otherwise</dt>
          <dd>
            <ol>
              <li>Set the source's <code><a href="#dom-mediasource-readystate" title="dom-MediaSource-readyState">readyState</a></code> attribute to <code><a href="#dom-mediasource-open" title="dom-MediaSource-OPEN">OPEN</a></code></li>
              <li>Fire a simple event named <code><a href="#dom-mediasource-openevent" title="dom-MediaSource-openEvent">open</a></code> at the source.</li>
              <li>Start a timer to call the <a href="#waiting-timeout">waiting timeout</a> algorithm after 350ms.</li>
              <li>Allow <a href="http://www.w3.org/TR/html5/video.html#concept-media-load-resource">resource fetch algorithm</a> to progress based on data passed into the source via <code><a href="#dom-mediasource-append" title="dom-MediaSource-append">append()</a></code></li>
            </ol>
          </dd>
        </dl>
      </li>
    </ol>

    <h3 id="algorithms-detach">4.2 Detaching from a media element</h3>
    <p>The follwing steps are run in any case where the media element is going to transition to <code>NETWORK_EMPTY</code> and fire an <code>emptied</code> event. These steps should be run right before the transition.
    <ol>
      <li>Set the source's <code><a href="#dom-mediasource-readystate" title="dom-MediaSource-readyState">readyState</a></code> attribute to <code><a href="#dom-mediasource-closed" title="dom-MediaSource-CLOSED">CLOSED</a></code></li>
      <li>Fire a simple event named <code><a href="#dom-mediasource-closeevent" title="dom-MediaSource-closeEvent">close</a></code> at the source.</li>
    </ol>


    <h3 id="waiting-timeout">4.3 Waiting Timeout</h3>
    <ol>
      <li>If the media element has received data from <code><a href="#dom-mediasource-append" title="dom-MediaSource-append">append()</a></code> and still has data to decode, abort these steps.</li>
      <li>Fire a simple event named <code><a href="#dom-mediasource-waitingevent" title="dom-MediaSource-waitingEvent">waiting</a></code> at the source.</li>
    </ol>

    <h3 id="seeking">4.4 Seeking</h3>
    <ol>
      <li>The media element <a href="http://www.w3.org/TR/html5/video.html#dom-media-seek">seeking algorithm</a> starts and has reached the stage where it has fired the <code>seeking</code> event on the media element</li>
      <li>If the source's <code><a href="#dom-mediasource-readystate" title="dom-MediaSource-readyState">readyState</a></code> attribute is set to <code><a href="#dom-mediasource-ended" title="dom-MediaSource-ENDED">ENDED</a></code>, then set it to <code><a href="#dom-mediasource-open" title="dom-MediaSource-OPEN">OPEN</a></code></li>
      <li>Start a timer to call the <a href="#waiting-timeout">waiting timeout</a> algorithm after 350ms.</li>
      <li>Fire a simple event named <code><a href="#dom-mediasource-flushevent" title="dom-MediaSource-flushEvent">flush</a></code> at the source.</li>
      <li>The media element <a href="http://www.w3.org/TR/html5/video.html#dom-media-seek">seeking algorithm</a> will proceed once data is passed to the media element via <code><a href="#dom-mediasource-append" title="dom-MediaSource-append">append()</a></code>.</li>
    </ol>
    <p class="note">
      This seeking algorithm assumes that the data passed to the MediaSource is packetized media with timestamps (ie Ogg pages or WebM clusters). This allows the "new playback position" to be determined by the timestamps in the data. If we want to support formats that are not packetized, like WAV files, then we will need a different mechanism to communicate the "new playback position" when a seek occurs.
    </p>

    <h2 id="examples">5. Examples</h2>
    <p>Example use of MediaSource</p>
    <div class="block">
      <div class="blockContent">
        <pre class="code">
&lt;script&gt;
  function onOpen(e) {
    var mediaSource = e.target;

    // Append the stream headers (ie WebM Header, Info, and Tracks elements)
    mediaSource.append(GetStreamHeaders());

    // Append some initial media data.
    mediaSource.append(GetNextCluster());
  }

  function onFlush(e) {
    var mediaSource = e.target;
    var video = document.getElementById('v');

    // Notify the cluster loading code to start fetching data at the 
    // new playback position.
    SeekToClusterAt(video.currentTime);

    // Append clusters from the new playback position.
    mediaSource.append(GetNextCluster());
    mediaSource.append(GetNextCluster());
  }

  function onProgress(e) {
    var mediaSource = e.target;

    if (mediaSource.readyState == mediaSource.ENDED)
      return;

    // If we have run out of stream data, then signal end of stream.
    if (!HaveMoreClusters()) {
      mediaSource.endOfStream();
      return;
    }

    // Is the next cluster larger than available buffer space?
    if (PeekNextCluster().length > mediaSource.bytesAvailable)
      return;

    mediaSource.append(GetNextCluster());
  }

  mediaSource = new MediaSource('video/webm; codecs="vp8, vorbis"');
  mediaSource.addEventListener('open', onOpen);
  mediaSource.addEventListener('flush', onFlush);
  mediaSource.addEventListener('progress', onProgress);
&lt;/script&gt;

&lt;video id="v" autoplay&gt; &lt;/video&gt;

&lt;script&gt;
  document.getElementById('v').src = URI.createObject(mediaSource);
&lt;/script&gt;
        </pre>
      </div>
    </div>
  </body>
</html>
